Sequential decision-making problems with multiple objectives arise naturallyin practice and pose unique challenges for research in decision-theoreticplanning and learning, which has largely focused on single-objective settings.This article surveys algorithms designed for sequential decision-makingproblems with multiple objectives. Though there is a growing body of literatureon this subject, little of it makes explicit under what circumstances specialmethods are needed to solve multi-objective problems. Therefore, we identifythree distinct scenarios in which converting such a problem to asingle-objective one is impossible, infeasible, or undesirable. Furthermore, wepropose a taxonomy that classifies multi-objective methods according to theapplicable scenario, the nature of the scalarization function (which projectsmulti-objective values to scalar ones), and the type of policies considered. Weshow how these factors determine the nature of an optimal solution, which canbe a single policy, a convex hull, or a Pareto front. Using this taxonomy, wesurvey the literature on multi-objective methods for planning and learning.Finally, we discuss key applications of such methods and outline opportunitiesfor future work.
展开▼
